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A color image processing apparatus for reproduces color 
data of a subject at higher accuracy by estimating the 
spectral reflectance of the subject from the color chip basis 
function vn (X) which can expand the spectral reflectances of 
all color chips of the reference chart and the expansion 
coefficients B of the color chips. The color image processing 
apararus uses an image processing device to which a subject 
shooting signal Q shot by the color image input device and 
a reference chart shooting signal G taken from a plurality of 
color chips each having a spectral reflectance is known. 
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COLOR IMAGE PROCESSING APPARATUS signals of three or more objects having independent XYZ 

values, and measured XYZ values. 

Usually, it is very difficult to accurately measure the 

BACKGROUND OF THE INVENTION spectral sensitivities of the input device, and therefore the 

i -5 conversion relationship from color image signals to XYZ 

Tne present invention relates to a color image processing valucg ^ usuaU obtamcd b ^ ^ * ^ 

apparatus which estimates the spectral reflectance of an Iq ^ cage wherc ^ xnsixiviiks of mc input 

object to be shot, from a color image signal. deyicc do ^ tfac ^ in a ^ 

As the color imaging apparatus becomes more popular, it accurate XYZ values cannot be obtained from a color image 

becomes more important to reproduce the original color of 10 signal for an arbitrary object. However, with regard to the 

an object exactly using an image display apparatus such as specific object, the conversion relationship can be obtained 

a CRT monitor or a printing device such as a printer, from by a way of least square method from the relationship 

a color image signal acquired by a color image signal input between the color image signals and XYZ values for a 

device (to be referred to simply as an "input device" number of colors of objects. 

hereinafter) such as a color scanner or a digital camera. 15 The format of the color chart used as the object for 

Further, as networks connecting a plurality of devices via obtaining color data from a color image signal acquired by 

wiring are widely used, the accurate transmission of the f c is standardized by ISO IT8.7, and provided 

color data of a color image signal and reproduction of color W 1 hlm makere * 

are becoming more important technical factors for estab- „ Id , order to f °btam the color data of an object under 

lishing systems for remote medical check-ups and exami- 20 dluounaton ^of observation-site different from the illumina- 

nations in the field of telemedicine or electronic shopping, Uon of shootmg-site used for acquiring a color unage sifma 

... • • ./j . , - rr -&» with respect to an arbitrary object, the product of the spectral 

which are expected to significantly expand from now. .cnsilivities of the input device and the shooting-she illu- 

In order to establish a system capable of reproducing an mination spectrum, and the product of the color matching 

accurate color, it is firstly required to estimate the color data functions and the observation -site illumination spectrum 

of an object accurately from a color image signal obtained 25 mU st have the relationship of linear conversions, 

from an input device. Such a condition depends upon the illumination spectrum, 

As a method of dealing with color quantitatively, the and therefore it is not practical in general. Consequently, in 
values of XYZ color system which is represented by the order to estimate the color data of an object under illumi- 
following equation (1) using functions of wavelength k, nation different from that of shooting-site, it is necessary to 
x(X), y(X) and z(X), corresponding to the spectral sensitivi- 30 obtaio me spectral reflectance of the object, 
ties of human vision, which is called as color matching For an accurate estimation of the spectral reflectance of an 
function as defined by Commission International de object, the method of obtaining the spectral reflectance of an 
TEclairage (CIE), and the spectrum of an object w(K), or a b y estimating the spectrum from a lot of multi- 
uniform color space based on the XYZ color system are channel L ima p s acquired by the spectral sensitivities of 
widelv used 35 narrow bands, and dividing the estimated spectrum by 

illumination spectrum can be proposed as disclosed in, for 

X 780 m example, Jpn. Pat. Appln. KOKAI Publication No. 

jUWAWA 9-172649. 

7 "^° Further, as disclosed in the Journal of Imaging Science 

y = k f y(AMA)dA 40 and Technology Vol, 40, No. 5, September/October 1996, 

*^ =3S0 p422-p430, in the case where objects to be shot are limited 

, f 780 to particular subjects such as human skin and the spectral 

Jx~m reflectance is represented by a linear combination of a small 

number of basis functions, it becomes possible to estimate 

45 the spectral reflectance from color image signals the band 

In the above equation, K is a constant. In order to obtain Qumber of which is equal to or more lhan lhe number of 

color data of an object accurately from a color image signal, basis ^ nclions representing the spectral reflectance of the 

it is essential to accurately estimate XYZ values of an object object 

under illumination of observation-site from the color image ^ ^ metho(J ^ ^ Qf ^ mm Qf ^ m ^ 

signal. In the case where the ulumination used for acquiring , rt J# . u i — • i - 
( 6 , . . , JL .n . . -,^. 6 50 nation used to acquire the color image signal is necessary in 

the color image signal and the ulumination of observation- , J% . 4 #u 4 , r.u • *a • 

, 5 , , . ... • . addition to the spectral sensitivities of the input device, 

site are equal to each other, the problem is to estimate the , c . A i 

XYZ values of the object under the Ulumination used for Howcve u r - lt 15 d f cult . ? or . ,he "P 6 "'?' ^ V 

acquiring the color image signal. mea ^ re the spectral sensitivity of the input device. In 

* , . . . . . , , consideration or a dinerence between mdividual input 

In this case in order to accurately obtain the XYZ values 5S devices> A a] ^ ^ Qr ^ ^ , hc acc q{ 

of an arbitrary object under the illumination used for obtain- me data j ded b ^ maker of thc device - a QOt a , w 

mg a color image signal, it is required that the spectral sufficient 

sensitivities of the input device and the color matching r, ' . , . . , . . 

c u u u i- * * . • i ■ Further, m order to obtain the data of the spectrum of the 

functions should have a linear conversion relationship. ... . - . . . # r 

, , . , e , . lllurmnation for the shooting environment, a measurement 

It is well known that the color image signal of an arbitrary 60 devke such as a spectrophotometer is requ ired when 

object can be accurately converted to XYZ values by linear shooXingy and therefor e these method cannot be easily 

conversion only if the above condition called a Luther Ued t0 a tem whkh j a convemional co i or 

condition is satisfied. ^ inpm device 

The conversion relationship can be obtained from the mn n 

relationship between the spectral sensitivities of the input 65 BRIEF SUMMARY OF THE INVENTION 

device and the color matching functions; however it can be The object of the present invention is to provide a color 

indirectly obtained from the relation between color image image processing device capable of estimating the spectral 
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reflectance of an object to be shot from the color image FIG. 13 is a diagram showing a detailed structure of an 

signal of the object acquired by a color image signal input image processing device in the color image processing 

device, even if the spectral sensitivities of the color image device according to the third embodiment; 

signal input device, or the spectrum of the illumination used . 

when the color image signal is input are unknown. 5 FIG ' 14 13 a diagram showing a structure of an object 

According to the present invention, there is provided a expansion coefficient calculating unit shown in FIG. 13; 

color image processing device including image processing FIG. 15 is a diagram showing a structure of a color image 

means for subjecting an image signal input of an object to be processing device according to the fourth embodiment of the 

shot, to a predetermined image processing, so as to be able present invention; 

to reproduce color data of the object, wherein the image io + £ . . . c 

processing means carries out the process of reproducing the fl FI °- 16 15 a , dl ^ nm .fT"""* a *™ ctu £ of a s P ectral 

color data of the object, by estimating the spectral reflec- reflectance evalua,m & un " shown m 15; 

tance of the object using the color image signal of a FIG. 17 is a diagram showing a structure of a color image 

reference object the spectral reflectance of which is known, processing device according to the fifth embodiment of the 

the color image signal of an object acquired by the same 15 present invention; 

illumination condition as that for acquiring the color image nG. 18 is a diagram snowing a concept of a format of 

signal of the reference object, and the spectral reflectance shot ^ da(a stored on a m Qa J md 
data or the reference object. 

Additional objects and advantages of the invention will be FIG - 19 is a diagram illustrating a color conversion 

set forth in the description which follows, and in part will be 20 P rocess tne fifth embodiment, 
obvious from the description, or may be learned by practice 

of the invention. The objects and advantages of the invention DETAILED DESCRIPTION OF THE 
may be realized and obtained by means of the instrumen- INVENTION 
talities and combinations particularly pointed out hereinaf- 
ter 25 Embodiments of the present invention will now be 

described with reference to drawings. 

BRIEF DESCRIPTION OF THE SEVERAL 5 

VIEWS OF THE DRAWING First, the concept of a color image processing device 

~ . 1 . . . ... according to the present invention will be provided. 

The accompanying drawings, which are incorporated in 6 * v 

and constitute a part of the specification, illustrate presently 30 According to the present invention, the spectral reflec- 

preferred embodiments of the invention, and together with tance of an object is estimated from the spectral reflectance 

the general description given above and the detailed descrip- of reference objects whose spectral reflectances are known 

tion of the preferred embodiments give below, serve to a priori and color image signal values of the reference 

explain the principles of the invention. objects and color image signal values of the object whose 

FIG. 1 is a diagram showing a structure of a color image 35 spectral reflectance is not known acquired from the color 

processing device according to the first embodiment of the image signal input device (to be called simply "input 

present invention; device" hereinafter). 

FIG. 2 is a diagram showing a detailed structure of an A mcthod of estimating the spectral reflectance of an 

image processing device in the color image processing fr om a color j mage> us i ng sucn an m ^ K dev i ce win 

device according to the first embodiment; 40 now be described in connection with an example case of a 

FIG. 3 is a diagram showing a structure of an object camera having the L-number of independent spectral sensi- 

expansion coefficient calculating unit shown in FIG. 2; tivities (bands). 

FIG. 4 is a diagram showing a structure of a spectral r , , , 

reflectance calculating unit shown in FIG. 2; u ^ 1 f f PpOSe herc ^ the ^f 1 fSf ^ ° f ^ 

r-T/i • j- i_ . flL , At band of the camera is represented by h (a.) (k=l to L). the 

FIG. 5A is a diagram showing a structure of the color 45 t c t , ... . f. r . • /\ . 

A , . 6 , . „ T/ ^ * spectrum of the illumination for shooting is represented by 

image input device shown in FIG. 1; ... „ . « . C4l _ r . • . ^ 

*L . , „ _ S(M, and the spectral reflectance of the object is represented 

FIG 5B is a diagram showing a structure of a filter turret b and {X) can be expanded by the L. DUmber 0 f basis 

to which interference filters are mounted and which are built functions elfX) 0=1 to D 
in the color image input device; 

FIG. 6 is a diagram showing a structure of an RGB value 50 Asignal value q* obtained by shooting an object under the 

calculating unit shown in FIG. 2; shooting-site illumination at the k-th camera sensitivity, is 

FIG. 7 is a diagram showing an example of a reference S iven b y the following formula supposing that the sensitivity 

chart used in the present invention; of | he camcra makcs a lincar response to the intensity of 

FIG. 8 is a diagram showing a structure of a color image 55 incident ^S^ 1 - 
processing device according to the second embodiment of 

the present invention; (2) 

FIG. 9 is a diagram showing a detailed structure of a . , , „ _ v c 

image processing device in the color image processing . Here > »~ the ff" 1 M ^V^ . p £> ofaD ° b f*, C ™ 

device according to the second embodiment; 60 be mt .° L-number of basis functions el(X) (1-1 

™~ 1A , , 4 ^ - . . 60 to L), p(X) is given by formula (3) using expansion coeffi- 

FIG. 10 is a diagram showing a structure of an object ri v nlc h /Li r 

expansion coefficient calculating unit shown in FIG. 9; 

FIG. 11 is a diagram showing a structure of a spectral L 

reflectance calculating unit shown in FIG. 9; p ^ _ ^cieti*) 

FIG. 12 is a diagram showing a structure of a color image 65 *-i 
processing device according to the third embodiment of the 
present invention; 



cients cl (1=1 to L). 

(3) 
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Therefore, formula (2) can be rewritten as: 







■fir ' 




(4) 


iUl- 
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= 
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-A, 





expect that: 

at-lej&WfaydX (5) 

10 

The signal values represented by equation (4) are obtained 
for the L-number of the sensitivities of the camera, the 
following matrix equation (6) is given: 











'c, • 






44- 


■ 4 




y, 




A4- 


■4. 


.CLj 



(6) 15 



The matrices of the equation (6) are represented by Q, A 
and C, respectively to give: 

Q-AC (7) 

Then, the estimated vector C of the expansion coefficient 
cl(l=l to L) of each basis function of spectral reflectance of 
the object is given by the following formula. 

C~A~ X Q (8) 

In the formula (8), Q is a matrix known by measurement, 
and therefore when A is available, the value for C can be 
obtained. When C is obtained, it becomes possible to obtain 
the spectral reflectance p(X) of the object from the formula 
(3). 

In order to obtain the value for A, the M-number of color 
chips whose spectral reflectances are known are shot. Sup- 
pose that the spectral reflectance of the m-th color chip is 
represented by fm(X) (m=>l to M). If the spectral reflectance 
fm(X) of the color chips can be expanded by the M-number 
of basis functions vn(X) (n=»l to M), fm(X) can be given with 
use of expansion coefficient bmn(n=l to M), by: 

« (9) 

n=l 

The shooting signal value g* 7 " of the m-th color chip by 
the k-th camera sensitivity, is given by: 

gS-lUVsWh'Wdk (10) 

From the formulas (9) and (10), the formula (11) is given 
by: 




(11) 



expect that: 

4.*-Jv»(X)s<X)**(X)dX (12) 

The signal value represented by equation (11) is obtained 65 
for the L-number of the sensitivities of the camera, the 
following matrix equation (13) is given: 



6 


















4, 







(13) 



The matrices of the equation (13) are represented by G, D 
and B, respectively to give: 

G-DB (14) 

From the equation (14), D can be given by: 

D-GB~ l (15) 

In the formula (15), since G is a measurement data and B 
is an expansion coefficient data corresponding to the basis 
functions of the pre-obtained spectral reflectances of the 
color chips, D can be obtained. 

Next, when the basis function el(X) (1=1 to L) of the 
spectral reflectance of the object is expanded by the basis 
functions vn(X) (n=l to M) of the spectral reflectances of the 
color chips, the basis function el(X) can be represented by 
the following formula using the expansion coefficient Oln. 

* (16) 
ettt) = 2j 0 <" v « (A > +(5 << A > 

In the above equation, b\(K) is an expansion error. When 
the formula (16) is substituted with the formula (5), the 
following formula is given: 

r (M } (17) 

fl{= J |X °tt v M + W) js(X)h l {X)d A 



If the formula (17) is represented as: 

A^O+A (18) 

In this equation, if term Acreated by expansion error 81(X) 
can be neglected, the estimated vector A' of A is given by: 

A '"DO (19) 

The estimated vector C of the expansion coefficient 
vector C of each basis function of the spectral reflectance of 
the object is given by: 

C = A'- l Q (20) 

= {DOT l Q 
= {GB- l Of l Q 

In this formula (20), Q represents a shooting signal of an 
object, O represents an expansion coefficient obtained by 
expanding the basis functions of the spectral reflectance of 
the object by the basis functions of the spectral reflectances 
of the color chips, B represents the expansion coefficient of 
the basis function of each spectral reflectance of the color 
chips, and G represents a shooting signal of the color chips. 
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By shooting a color chips whose spectral reflectances are When 

known, G is obtained. Thus, from the formula (20), with «'<<X)-*X*>oW (26) 

values of an expansion coefficient B of the basis functions of y , / 27) 

each spectral reflectance of the color chips which are \ . 

obtained in advance, an expansion coefficient 0 of the basis 5 and el (X) is expanded by vn'(X), cV(\) can be represented, 

functions of the spectral reflectances of an object which is Wth Use of ex P aQS10D coefficient O'ln, as: 

calculated in advance and a shooting signal Q of an object, w 

the expansion coefficient of the basis functions for the e / (AJ _ y oJXW + W 

spectral reflectance of an object can be obtained. Then, from «-i 

the basis functions for the spectral reflectance of the object 10 

and the expansion coefficient C, the spectral reflectance of « iL - . ,- QX , Mox 

the object is calculated. From ^ formulas ^ and 

In particular, when the basis functions el(X) of the spectral „ 

reflectance of an object and the basis functions vn(X) of A - J e^A^WX 

color chips coincide with each other, O becomes a unit 15 

matrix, and the estimated vector C of C is given by: = Jef(A)A*(A)rfA 

c=(G5" 1 )- 1 c? (21) 



C=A" l Q (32) 



As represented by the formula (21), when the basis 20 

functions for the spectral reflectances of the color chips and " r r 

the basis function of the spectral reflectance of the object = L<J < m + J (WA 
coincide with each other, the spectral reflectance of the 

object can be obtained from the shooting signal Q of the - YV +a' 

signal using a color chip shooting image signal G shot by a 25 ~ j£f ° ' 
camera having the same number of bands as the number of 
the basis functions. 

When the spectral reflectance of an object can be * 8 iven - ^ formula < 29 ) can be rewritten » : 

expanded by the basis functions of the color chips, the a^do'+a (30) 

spectral reflectance of the object can be obtained as a linear 30 i n m is equation, if term A* created by expansion error 

combination of the basis functions of the spectral reflec- 6'1(X) can be neglected, the estimated vector A'of A is given 

tances of the color chips, by rendering C of the formula (21) by: 

the expansion coefficient of the basis functions of the a'-DO' (31) 

spectral reflectance of the color chips. 

f *u u • «. *j c *u * i 35 The estimated vector C of the expansion coefficient 

In this case, even if the basis functions of the spectral * ^ r u u • t * ■ r*i_ * i a . r 

„ . r u- . • .i ( l . i a . vector C of each basis function of the spectral reflectance of 

reflectance of an object is not known, the spectral reflectance the ob ect is Ven b 

of an object can be obtained by a camera having bands 

whose number is equal to or more than that of the basis 

functions of the spectral reflectances of the color chips. 40 

Further, if the basis functions of the spectral reflectances of _ ^do')~ 1 q 

the object are known, and it can be expanded by the basis 

functions of the spectral reflectances of the color chips, the = (GB~ X &)~ Q 
spectral reflectance of the object can be obtained by the 

formula (20) from the shooting signal Q of the object, using 45 ^ formu , a p } &h ^ 

the color chip shooting image signal G taken by a camera Qbj Q , ^ an P ansion wfScien f obtained by 

having bands whose number is equal to or more than that of . , q . . . . t c L . , ' 

, , 5 r . , Z 1 expanding the product of the basis functions of the spectral 

the basis functions of the object. A . r u- . j .u u- * u *• ■« 

J reflectance of an object and the object shooting-site lllumi- 

Next, it is the case where different illumination conditions nation spectrum, of the formula (28) by the products of the 

are used for the shooting of the color chips and the object. 50 basis functions of the color chips and the color chip 

When the illumination spectrum at the shooting of an shooting-site illumination spectrum, B represents the expan- 

object is So(X), the formula (2) can be rewritten as: sion coefficient of the basis function of each spectral reflec- 
tance of the color chips, and G represents a shooting signal 

f-l&)sJto#(Wk (22) of the color chips. 

55 By shooting color chips whose spectral reflectances are 

Further, the formula (5) can be expressed as: known, G is obtained. Thus, from the formula (32), with 

values of an expansion coefficient B of the basis functions of 

«/-JtfAKM A *W^ (23) each spectral reflectance of the color chips which are 

obtained in advance, a pre-calculated O' and a shooting 

When the illumination spectrum at the shooting of color 60 signal Q of an object, the expansion coefficients of the basis 

chips is Sc(X), the formula (10) can be rewritten as: functions for the spectral reflectance of an object can be 

obtained. 

*»M/»(*K(*) fc *(*> 0 ' ( 24 ) Then, the spectral reflectance of the object can be calcu- 
lated from its basis functions and expansion coefficients. 

Further, the formula (12) can be expressed as: 65 With the present invention, even in the case where the 

spectral sensitivities of the camera and the spectrum of the 

4i*-KM»c(*)a*M^ (25) shooting-site illumination are unknown, the spectral re flee - 
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tance of an object is estimated from the shooting signal of 
the object, having the basis functions of the spectral 
reflectance, by which the basis function of the spectral 
reflectance of the object can be expanded. 

Further, even in the case where the illumination condition 
for shooting the color chips and the illumination condition 
for shooting the object are different from each other, the 
spectral reflectance of the object can be estimated from the 
shooting signal of the object if the color chip shooting-site 
illumination spectrum and object shooting-site illumination 
spectrum are known. 

In the estimation of the spectral reflectance of an object 
described above, there are conditions which the spectral 
reflectances of color chips in reference object whose spectral 
reflectance are known must satisfy in order to accurately 
estimate the spectral reflectance of the object. By forming in 
advance an object which satisfies these conditions, more 
accurate estimation of the spectral reflectance can be per- 
formed than in the case where a conventional reference 
object is used. 

In connection with the above -described spectral reflec- 
tance estimating method, it is explained that the number of 
basis functions of the spectral reflectances of the color chips 
whose spectral reflectance are known, coincides with that of 
color chips; however a method similar to this can be applied 
to cases where they are different. 

Further, it is supposed in the above description that the 
number of basis functions of spectral reflectances of an 
object and the number of bands of the input device coincide 
with each other; however this method can be expandable for 
cases where these numbers are different from each other, For 
example, as in the case of an ordinary digital camera, the 
method can be applied to a 3 -band input device. 

As a reference object whose spectral reflectance is known, 
the color chip has been discussed as an example; however 
arbitrary objects can be applied to this method as long as 
they have spectral reflectances similar to those of the color 
chips. 

FIG. 1 is a diagram showing an example of the structure 
of a color image processing apparatus according to the first 
embodiment of the present invention, which will now be 
described. 

The apparatus of this embodiment, includes a color image 
input device (input device) 1, a reference chart 2 and an 
object 3, arranged under the same illumination A, an 
observation-site illumination spectrum data storage unit 4, 
an image processing device 5 and a CRT monitor 6. 

The image processing device 5 shoots the reference chart 
2 and the object 3 arranged under the same illumination A in 
order or at the same time by the input device 1. Further, the 
spectral reflectance and XYZ values of the object 3 are 
estimated from the image signals G and Q output, and the 
basis functions and expansion coefficients data of the reflec- 
tances of color chips, and color matching functions data, 
which are present in advance within the image processing 
device, and thus RGB signals are output. The RGB signals 
are displayed on the CRT monitor 6 as a color image of the 
object. 

In this embodiment, the XYZ values under the 
observation-site illumination are obtained from the spectral 
reflectance of the object, which is estimated by the image 
processing device 5, and further the data is displayed on the 
CRT monitor as a color image. However, the present inven- 
tion is not limited to this embodiment, but is applicable to 
the system including display devices such as a liquid crystal 
display device and a plasma display, and a printing device 
such as a color printer. Further, it is possible to store the 
spectral reflectance and XYZ values as data. 
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With the color image processing apparatus having the 
above-described structure, if the reference chart 2 and the 
object 3 are shot at the same time by the input device 1, the 
image signal of the reference chart 2 is extracted from the 
5 object shooting signal by a reference chart extracting device 
(not shown), in the image processing device 5, 

The input device 1 forms an image on a semiconductor 
imaging device such as a CCD 104, from the transmitted 
light through a camera lens 1 and one of a plurality of 
10 interference filters 103 mounted on the filter turret 102. 

These interference filters 103 are mounted on the filter 
turret 102 having a circular shape, as schematically shown 
in FIG. 5B. As the filter turret 102 is rotated by the motor 
105, each of the interference filters 103 is mounted on the 
is rear surface of the camera lens 101 in order, and an image 
of the corresponding band is shot by the CCD 104. The CCD 
104 has the pixel number of, for example, 640 pixelsx480 
pixels. Naturally, the present invention is not limited to this 
pixel number. 

20 The output signal of an image shot by the CCD 104 is sent 
to the image processing device 5. 

FIG. 7 shows an example of the reference chart 2. The 
reference chart 2 consists of 10 color chips, and the spectral 
reflectance of each color chip can be expanded by 10 basis 

25 functions which are independent from each other. The image 
signals G and Q acquired by the input device 1 are input to 
the image processing device 5 as 10 channels image data 
having a pixel size of 640 pixelsx480 pixels of 10-bits per 
pixel. 

30 To the image processing device 5 to which the image 
signals G and Q have been input, observation-site illumina- 
tion spectrum data is input from the observation-site illu- 
mination spectrum data storage unit 4. The observation -site 
illumination spectrum data consists of light intensity values 

35 of observation-site illumination taken at an interval of 1 nm, 
in a wavelength range from 380 nm to 780 nm. 

In the image processing device 5, the spectral reflectance 
and the XYZ values of the object are estimated from these 
input data, and the color chip basis function data, color chip 

40 expansion coefficient data and color matching functions 
data, which are pre-stored in the image processing device. 
Further, the image processing device 5 converts, on the basis 
of the characteristics data of the CRT monitor 6, the XYZ 
values of the object 3 taken under the observation-site 

45 illumination into RGB signals to be displayed on the moni- 
tor screen, and outputs the signals to the CRT monitor 6. 

As the RGB signals output from the image processing 
device are input to the CRT monitor 6, the CRT monitor 6 
displays the color image of the object. 

50 Next, the structure of the image processing device 5 of 
this embodiment will now be described with reference to 
FIG. 2. 

The image processing device 5 consists mainly of a 
spectral reflectance calculating unit 7, a tristimulus value 
55 calculating unit 8 and an RGB value calculating unit 9. 

The spectral reflectance calculating unit 7 comprises of a 
color chip shooting signal storage unit 11, a color chip 
expansion coefficient storage unit 12, an object expansion 
coefficient calculating unit 10, a color chip basis function 
60 storage unit 14 and a spectral reflectance calculating unit 13 
for calculating a spectral reflectance p(X). 

The color chip shooting signal storage unit 11 calculates, 
from the shooting image of the reference chart containing a 
plurality of color chips, the average values of shooting signal 
65 values corresponding to the color chips, for an appropriate 
region, and stores them as color chip shooting signals. The 
color chip expansion coefficient storage unit 12 stores 
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expansion coefficients corresponding to basis functions of 
the spectral reflectances of the color chips in the reference 
chart, which are obtained in advance. 

The object expansion coefficient calculating unit 10 cal- 
culates an expansion coefficient C of the spectral reflectance 
of an object from the object shooting signal input from the 
color image input device 1, the color chip shooting signal G 
input from the color chip shooting signal storage unit 11 and 
the expansion coefficient B input from the color chip expan- 
sion coefficient storage unit 12. 

Further, the color chip basis function storage unit 13 
stores color chip basis functions vn(X) (n-1 to 10), and the 
spectral reflectance calculating unit 13 calculates the spec- 
tral reflectance p(X) of an object from the expansion coef- 
ficient C calculated by the object expansion coefficient 
calculating unit 10 and the basis functions vn(\) (n=l to 10) 
of the object output from the color chip basis function 
storage unit 14. 

FIG. 3 shows the structure of the object expansion coef- 
ficient calculating unit 10, which will now be described. 

An inverse matrix B" 1 of the expansion coefficient B 
calculated by an inverse matrix operating unit A19, from the 
expansion coefficient B input from the color chip expansion 
coefficient storage unit 12, and a color chip shooting signal 
G input from the color chip shooting signal storage unit 11 
are input to a matrix multiplication unit A20, and thus a 
matrix (GB" 1 ) is calculated. The calculated GB" 1 is input to 
an inverse matrix operating unit B21 to obtain an inverse 
matrix thereof (GB" 1 )" 1 , which is input to a matrix multi- 
plication unit B22. To the matrix multiplication unit B22, the 
object shooting signal Q is further input, and the expansion 
coefficient G=(GB _1 ) _1 Q is calculated from (GB -1 )" 1 output 
from the inverse matrix operating unit B21. 

FIG. 4 shows the structure of the spectral reflectance 
calculating unit 13, which will now be described. 

The spectral reflectance calculating unit 13 consists of a 
basis coefficient multiplication unit 23 for multiplying an 
object expansion coefficient C corresponding to a color chip 
basis function vn(k), and a basis function integrating unit 24 
for calculating the spectral reflectance. 

The basis coefficient multiplication unit 23 obtains, by 
multiplication, the product of object expansion coefficients 
C=Cn (n=l to 10) corresponding to the color chip basis 
functions vn(X), from the object expansion coefficient C 
input from the object expansion coefficient calculating unit 
10 and the color chip basis function vn(X) input from the 
color chip basis function storage unit 14, and thus obtained 
result is output to the basis function integrating unit 24. 

The basis function integrating unit 24 adds up the data 
input from the basis coefficient multiplication unit 23 at all 
the wavelengths in terms of all the basis functions cnvn(X) 
(n-1 to 10), and thus the spectral reflectance is calculated. 

In the tristimulus value calculating unit 8 shown in FIG. 
2, the tristimulus value operating unit 16 calculates the 
tristimulus values XYZ under the illumination condition for 
observing an object, from the spectral reflectance of the 
object, calculated from the spectral reflectance calculating 
unit 7, the observation-site illumination input from the 
observation-site illumination spectrum data storage unit 4, 
and the color matching function data pre-stored in the color 
matching function data storage unit 15, and the values are 
output to the RGB value calculating unit 9. 

The RGB value calculating unit 9, as shown in FIG. 6, 
comprises a CRT monitor profile storage unit 17 for storing 
the characteristics data of the display device and an output 
signal calculating unit 18 for converting the XYZ values to 
the monitor input signal RGB values using the characteris- 
tics data of the CRT monitor. 
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The characteristics data of the CRT monitor contains a 
matrix M stored in the phosphor tristimulus value data 
storage unit 25 and a gamma curve /Y'v* stored in the RGB 
gradation characteristics data storage unit 26. The matrix M 
5 is a matrix made of elements of 3x3, and the gamma curves 
yVY* ^ tne output luminance value to each of the RGB 
input values. 

In the RGB value calculating unit 9, the XYZ value 
calculated by the tristimulus values calculating unit 8 and 

10 the matrix M stored in the phosphor tristimulus value data 
storage unit 25 are multiplied by the matrix multiplication 
unit 27. Then, the gradation correcting unit 28 performs the 
correction of the gradation on the basis of the result of the 
multiplication and the gamma curves stored in the RGB 

15 gradation characteristics data storage unit 26, and thus the 
RGB signals are obtained by conversion. 

Thus, the calculated RGB signals are input to the CRT 
monitor, and a color image of the object is displayed on the 
CRT monitor. 

20 According to the first embodiment of the present 
invention, the spectral reflectance of an object is estimated 
from the object shooting signal Q and reference chart 
shooting signal G shot under the same illumination condi- 
tion by the color image input device, and the color chip basis 

25 functions vn(X), which can expand the spectral reflectance of 
the spectral reflectance of all the color chips of the reference 
chart pre-stored, (that is, the spectral reflectance data of the 
object) and the expansion coefficients B of the color chips. 
Here, the conditions for being able to estimate the spectral 

30 reflectance of an object exactly are that the spectral reflec- 
tance of the object can be expanded by the color chip basis 
functions vn(X) of the reference chart and the color image 
input device has the number of bands which is equal to or 
more than that of color chip basis functions. With these 

35 conditions, the spectral reflectance of an object can be 
estimated even if the spectral sensitivities of the color image 
input device and the illumination spectrum used for shooting 
are unknown. 

FIG. 8 shows a color image processing apparatus accord- 

40 ing to the second embodiment of the present invention. 
The apparatus of this embodiment consists of an input 
device 1 to which a color image is input by shooting, a 
reference chart 2 and an object 3, which are arranged under 
the same illumination A, an observation-site illumination 

45 spectrum data storage unit 4 for storing observation -site 
illumination spectrum data, an object basis function storage 
unit 29 for storing object basis function el(X), an image 
processing device 30 for outputting RGB signals and a CRT 
monitor 6 far displaying a color image. 

50 The image processing device 30 shoots the reference chart 
2 and object 3, which are exposed by the same illumination 
A sequentially or at the same time, by the input device 1, and 
estimates the spectral reflectance and XYZ values of the 
object 3 from the shooting signals G and Q of those shot 

55 respectively, and the color chip basis functions data, color 
chip expansion coefficients data and color matching func- 
tions data, pre-stored in the image processing device. Thus, 
the RGB signals are output. The RGB signals are displayed 
on the CRT monitor 6 as a color (RGB) image of the object. 

60 The input device 1 is equivalent to the input device of the 
first embodiment described above, and a filter turret on 
which interference filters are arranged, as shown in FIGS, 
5 A and 5B, is mounted. Further, the reference chart of this 
embodiment has the same structure as shown in FIG. 7. 

65 Therefore, the explanations therefor will not be repeated. 
To the image processing device 30, the image signals G 
and Q shot by the input device 1 are input, the observation- 
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site illumination spectrum data is input from the 
observation-site illumination spectrum data storage unit 4, 
and the basis function data of the object is input from the 
object basis function storage unit 29. The observation-site 
illumination spectrum data consists of intensity values of 
observation-site illumination taken at an interval of 1 nm, in 
a wavelength range from 380 nm to 780 nm. 

In the image processing device 30, the spectral reflectance 
and XYZ values of the object are estimated from these input 
data, the color chip expansion coefficient B pre -stored in the 
image processing device, the expansion coefficient O 
obtained by expanding the basis function of an object by the 
basis functions of the color chips, the color chip basis 
functions vn(X), and the color matching functions data. 
Further, RGB signals to be input to the CRT monitor, are 
output; the RGB signals are used for displaying the XYZ 
values of the object taken under the observation-site 
illumination, on the CRT monitor, using the characteristics 
data of the CRT monitor. The CRT monitor displays a color 
image of the object, as the RGB signal output from the 
image processing device is input thereto. 

lie structure of the image processing device will now be 
described in detail. 

As shown in FIG. 9, the image processing device 30 
consists mainly of a spectral reflectance calculating unit 31, 
a tristimulus value calculating unit 8 and an RGB value 
calculating unit 9. Here, the structural elements of this 
embodiment, which are similar to those shown in FIG. 2, 
will be designated by the same reference numerals, and the 
detailed descriptions therefor will not be repeated. 

The spectral reflectance calculating unit 31 includes a 
color chip shooting signal storage unit 11 for storing a color 
chip shooting signal, a color chip expansion coefficient 
storage unit 12 for storing an expansion coefficient B, a color 
chip basis function storage unit 14 for storing color chip 
basis functions vn(A) (n=l to 10), an object basis expansion 
coefficient calculating unit 32 for calculating a expansion 
coefficient O, an object expansion coefficient calculating 
unit 33 for calculating an expansion coefficient C, and a 
spectral reflectance calculating unit 13 for calculating a 
spectral reflectance p(X) of an object. 

In the spectral reflectance calculating unit 31 having the 
above described structure, the object shooting signal Q and 
reference chart shooting signal G output from the color 
image input device 1 are input to the object expansion 
coefficient calculating unit 33 and the color chip shooting 
signal storage unit 11 respectively. In the color chip shooting 
signal storage unit 11, from a reference chart image made of 
a plurality of color chips, average values of the shooting 
signals for all the color chips are calculated in an appropriate 
region, and the calculation result is stored as a color chip 
shooting signal. In the color chip expansion coefficient 
storage unit 12, expansion coefficients for the basis functions 
of the color chips within a reference color chip obtained in 
advance are stored. 

In the object basis expansion coefficient calculating unit 
32, on the basis of the color chip basis function vn(X) input 
from the color chip basis function storage unit 14 and the 
object basis functions el(X) input from the object basis 
function storage unit 29, the expansion coefficient O in the 
case of expanding the object basis function by the color 
basis function vn(X) is obtained. 

In the object expansion coefficient calculating unit 33, on 
the basis of the object shooting signal Q input from the color 
image input device 1, the color chip shooting signal G input 
from the color chip shooting signal storage unit 11, the 
expansion coefficient B input from the color expansion 
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coefficient storage unit 12 and the expansion coefficient O 
input from the object basis expansion coefficient calculating 
unit 32, the expansion coefficient C (G^GB^O^Q) in the 
case of expanding the reflectance of an object by the object 

5 basis function el(X) is calculated, and the calculation result 
is output to the spectral reflectance calculating unit 13, The 
spectral reflectance calculating unit 13 calculates the spec- 
tral reflectance p(X) of the object from the expansion coef- 
ficient C and the basis functions el(X) output from the object 

10 basis function storage unit 29. 

FIG. 10 shows the structure of the object expansion 
coefficient calculating unit 33, which will now be described. 

The inverse matrix B" 1 of the expansion coefficient B, 
which is calculated by the inverse matrix operating unit A34 

15 from the expansion coefficient B input from the color chip 
expansion coefficient storage unit 12, the color chip shooting 
signal G input from the color chip shooting signal storage 
unit 11, and the expansion coefficient O input from the 
object basis expansion coefficient calculating unit 32 are 

20 input to the inverse Matrix Multiplication unit A35, and thus 
matrix GB _1 0 is calculated. The obtained matrix GB" J 0 is 
input to the inverse matrix operating unit B36, and thus an 
inverse matrix (GB^O)" 1 is obtained, which is further input 
to the matrix multiplication unit B37. 

25 To the matrix multiplication unit B37, the object shooting 
signal Q is further input. With this input, and the inverse 
matrix (GB" 1 0)" 1 output from the inverse matrix operating 
unit B36, the expansion coefficient of the object, 
C^GB^O)"^ is calculated, and the result of the calcula- 

30 tion is output to the spectral reflectance calculating unit 13, 
As shown in FIG. 9, in the spectral reflectance calculating 
unit 13, the spectral reflectance p(X) of the object is calcu- 
lated on the basis of the expansion coefficient C calculated 
from the object expansion coefficient calculating unit 33 and 

35 the basis functions el(>.) of the object, which is output from 
the object basis function storage unit 29. 

FIG. 11 shows the structure of the spectral reflectance 
calculating unit 13, which will now be described. 
The spectral reflectance calculating unit 13 includes a 

40 basis coefficient multiplication unit 38 for multiplying the 
object basis functions el(X) with the corresponding object 
expansion coefficient C, and a basis function integrating unit 
24 for calculating the spectral reflectance. 

With the object expansion coefficients C input from the 

45 object expansion coefficient calculating unit 33 and the 
object basis functions el(X) input from the object basis 
function storage unit 29, the basis coefficient multiplication 
unit 38 multiples the object basis functions el(X) and the 
object expansion coefficients C«=cl (1=1 to 10) respectively, 

50 and the result of the calculation is output to the basis 
function integrating unit 24. The basis function integrating 
unit 24 adds up the data input from the basis coefficient 
multiplication unit 38 with regard to all of the basis func- 
tions clel (X) (1-1 to 10) for each wavelength, and the result 

55 is output to the tristimulus value calculating unit 8. 

In the tristimulus value calculating unit 8 shown in FIG. 
9, the tristimulus value operating unit 16 calculates the 
tristimulus values XYZ under the illumination condition for 
observing the object on the basis of the spectral reflectance 

60 of the object, calculated in the spectral reflectance calculat- 
ing unit 13, the observation-site illumination spectrum data 
input from the observation-site illumination spectrum data 
storage unit 4 and the color matching functions data pre- 
stored in the color matching function data storage unit 15. 

65 As shown in FIG. 6, the output signal calculating unit 18 
converts the XYZ values calculated in the tristimulus value 
calculating unit 8 into an RGB signals to be input to the 
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monitor, with use of the characteristics data of the CRT like, a reference chart 2a placed under illumination A, an 

monitor, pre -stored in the CRT monitor profile storage unit object 3a placed under illumination B different from the 

17. illumination A, an observation-site illumination spectrum 

The characteristics data of the CRT monitor includes a data storage unit 4 for storing observation-site illumination 

matrix M stored in the phosphor tristimulus value data 5 spectrum data, an object basis function storage unit 29 for 

storage unit 25 shown in FIG. 6 and gamma curves Y*y G Y* storing an object basis function el(X), an object shooting-site 

stored in the RGB gradation characteristics data storage unit Ulumination spectrum data storage unit 39 for storing object 

26. shooting-site illumination spectrum data, which will be 

The matrix M is a matrix consisting of elements arranged explained later, a reference chart shooting-site illumination 
in 3x3, and the gamma curves y^^Y^ output light 10 spectrum data storage unit 40 for storing reference chart 
intensity to respective input RGB values. The RGB signals shooting-site illumination spectrum data, an image process- 
output from the RGB value calculating unit 9 are input to the ing device 41 for estimating a spectral reflectance and an 
CRT monitor 6, and the color image of the object is XZY values of an object 3a from image signals G and Q of 
displayed on the CRT monitor 6. the reference chart 2a and object which are shot, and for 

In the second embodiment, the spectral reflectance of an 15 output ting RGB signals, and a CRT monitor 6 for displaying 

object is estimated from the object shooting signal Q and the XYZ values of the object under the observation-site 

reference chart shooting signal G, shot by the color image illumination, from the RGB signals, 

input device, the color chip basis functions vn(^) which can The input device 1 is equivalent to the input device of the 

expand the spectral reflectance of every color chip of the first embodiment, and a filter turret on which interference 

reference chart pre-stored, the expansion coefficient B of 20 filters shown in FIGS. 5A and SB are arranged is mounted 

each color chip and the basis function el(K) of the spectral in the input device 1. Further, this embodiment has an 

reflectance of the object. equivalent structure to that shown in FIG. 7 in terms of the 

As in the case of the first embodiment, the conditions for reference chart, and therefore the explanations therefor will 

being able to estimate the spectral reflectance of an object not be repeated. The illumination spectrum data are intensity 

without error are that the basis functions el(X) of the spectral 25 values of illumination taken at an interval of 1 nm within a 

reflectance of an object can be expanded by the color chip wavelength range of 380 nm to 780 nm. The basis functions 

basis functions vn(\) of the reference chart, and that the data of the object are input from the object basis function 

color image input device has a certain number of bands, storage unit 29. 

which is more than the number of basis functions of the The image processing device 41 estimates the spectral 

spectral reflectances of the object. With these conditions, 30 reflectance and XYZ values of an object from the above - 

even if the spectral sensitivities of the color image input described inputs, the color chip expansion coefficient B 

device and the illumination spectrum for shooting are not pre-stored in the image processing device, the expansion 

known, the spectral reflectance of the object can be esti- coefficient O' obtained by expanding the products of the 

mated. basis functions of the object and the object shooting-site 

It should be noted that the second embodiment is different 35 illumination spectrum by the products of the basis functions 

from the first one in that the number of bands which is of the color chips and the reference chart shooting-site 

required by the color image input device in order to accu- illumination spectrum, the basis functions el(K) of the object 

rately estimate the spectral reflectance of the object, is not and the color matching function data. Further, with reference 

the number of color chip basis functions, but the number of to the characteristics data of the CRT monitor 6, the device 

basis functions of the reflectance of the object. 40 41 outputs the XYZ values of the object under the 

In general, the number of basis functions of the spectral observation-site illumination, as RGB signals, to the CRT 

reflectance of the object is less than the number of basis monitor 6. The CRT monitor 6 displays the color image of 

functions of the spectral reflectances of the color chips, the object on the basis of the RGB signals, 

which can expand the basis function of the spectral reflec- FIG. 13 shows an example of the structure of the image 

tance of the object. Therefore, it becomes possible to esti- 45 processing device 41, which will now be described, 

mate the spectral reflectance of the object by a camera The image processing device 41 consists mainly of a 

having the less number of bands than that of the first spectral reflectance calculating unit 42, a tristimulus value 

embodiment. Further, a great number of color chips can be calculating unit 8 and an RGB value calculating unit 9. Here, 

used to be able to expand the spectral reflectance of an object the structural elements of this embodiment, which are simi- 

at high accuracy, while maintaining the number of bands of 50 lar to those shown in FIG. 9, will be designated by the same 

the camera for shooting an object equal to the number of . reference numerals, and the detailed descriptions therefor 

basis functions of the spectral reflectance of the object. With will not be repeated. 

use of the same method as in this embodiment, it is possible The spectral reflectance calculating unit 42 includes a 

to accurately estimate the spectral reflectance of an object color chip shooting signal storage unit 11 for storing a color 

such as human skin, whose spectral reflectance can be 55 chip shooting signal, a color chip expansion coefficient 

reproduced by three basis functions at high accuracy, in the storage unit 12 for storing an expansion coefficient B, a color 

case of an ordinary three-band digital camera, as presented chip basis function storage unit 14 for storing color chip 

in the fifth embodiment. basis functions vn(X) (n-1 to 10), an object basis expansion 

FIG. 12 shows the structure of the color image processing coefficient calculating unit 43 for calculating a expansion 

apparatus according to the third embodiment of the present 60 coefficient 0\ an object expansion coefficient calculating 

invention, which will now be described. unit 44 for calculating an expansion coefficient C, and a 

It should be noted that the structural elements of this spectral reflectance calculating unit 13 for calculating a 

embodiment which are equivalent to those shown in FIG. 8 spectral reflectance p(X) of an object from the expansion 

are designated by the same reference numerals, and the coefficient C and the color chip basis functions vn(X) output 

detailed explanations therefor will be omitted. 65 from the object basis function storage unit 29. 

The device of this embodiment includes a color image The object basis expansion coefficient calculating unit 43 

input device 1 for inputting a color image by shooting or the calculates an expansion coefficient O 1 of the case where the 
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products of the object basis functions el(X) and the object 
shooting-site illumination spectrum, that is, el(X)So(X), is 
expanded by the products of the color chip basis functions 
vn(X) and the reference chart shooting-site illumination 
spectrum, that is, vn(X)*Sc(X). 5 

The object expansion coefficient calculating unit 44 cal- 
culates an expansion coefficient C (C'=(GB" 1 0 , )~ 1 Q) on the 
basis of the object shooting signal Q, the color chip shooting 
signal G, the expansion coefficient B and the expansion 
coefficient 0\ 10 

FIG. 14 shows the structure of the object expansion 
coefficient calculating unit 44, which will now be described. 

Here, the structural elements of this embodiment, which 
are similar to those shown in FIG. 10, will be designated by 
the same reference numerals, and the detailed descriptions is 
therefor will not be repeated. 

The inverse matrix B" 1 of the expansion coefficient B, 
which is calculated by the inverse matrix operating unit A34 
from the expansion coefficient B input from the color chip 
expansion coefficient storage unit 12, the color chip shooting 20 
signal G input from the color chip shooting signal storage 
unit 11, and the expansion coefficient O' input from the 
object basis expansion coefficient calculating unit 43 are 
input to the matrix multiplication unit A45, and thus matrix 
GB -1 0' is calculated. The obtained matrix GB^O' is input 25 
to the inverse matrix operating unit B36, and thus an inverse 
matrix (O(GB -1 0')~ 3 ) is obtained, which is further input to 
the matrix multiplication unit B37. 

To the matrix multiplication unit B37, the object shooting 
signal Q is further input. With this input, and the inverse 30 
matrix (GB^O 1 )" 1 output from the inverse matrix operating 
unit B36, the expansion coefficient of the object, C is 
calculated, and the result of the calculation is output to the 
spectral reflectance calculating unit 13. 

As shown in FIG. 13, in the spectral reflectance calculat- 35 
ing unit 13, the spectral reflectance p(X) of the object is 
calculated on the basis of the expansion coefficient C 
calculated from the object expansion coefficient calculating 
unit 44 and the basis functions el(X) of the object, which are 
output from the object basis function storage unit 29. Then, 40 
the result of the calculation is output to the tristimulus value 
calculating unit 8. 

It should be noted that the spectral reflectance calculating 
unit 13 has the same structure as illustrated in FIG. 11, and 
therefore the explanation of this unit will not be repeated. 45 

The tristimulus value calculating unit 8 calculates the 
tristimulus values XYZ under the illumination condition for 
observing the object 3a on the basis of the spectral reflec- 
tance of the object, calculated in the spectral reflectance 
calculating unit 42, the observation-site illumination spec- 50 
trum data input from the observation-site illumination spec- 
trum data storage unit 4 and the color matching functions 
data pre-stored in the color matching function data storage 
unit 15. Then, the calculated value is output to the output 
signal calculating unit 18. 55 

The output signal calculating unit 18 converts the XYZ 
values calculated in the tristimulus value calculating unit 8 
into RGB signals to be input to the monitor, with use of the 
characteristics data of the CRT monitor 6, pre-stored in the 
CRT monitor profile storage unit 17. Thus, the color image 60 
is displayed on the CRT monitor 6. 

In the third embodiment, the spectral reflectance of an 
object is estimated from the object shooting signal Q and 
reference chart shooting signal G, shot by the color image 
input device under different illumination conditions, the 65 
color chip basis functions vn(X) which can expand the 
spectral reflectance of every color chip of the reference chart 



pre-stored, the expansion coefficient B of each color chip, 
the basis function of the spectral reflectance of the object, 
the reference chart shooting-site illumination spectrum data 
and the object shooting-site illumination data. 

Here, the conditions for being able to estimate the spectral 
reflectance of an object without error are that the products of 
the basis functions of the spectral reflectance of the object 
and the object shooting-site illumination spectrum, can be 
expanded by the products of the color chip basis functions 
vn(X) of the spectral reflectance of reference chart and the 
reference chart shooting -site illumination spectrum, and that 
the color image input device has a certain number of bands, 
which is more than the number of basis functions of the 
spectral reflectance of the object. 

With these conditions, even if the spectral sensitivities of 
the color image input device are not known, the spectral 
reflectance of the object can be estimated. The third embodi- 
ment is different from the second one mainly in the follow- 
ing point. That is, even if the reference chart shooting -site 
illumination and the object shooting-site illumination are 
different from each other, the spectral reflectance of an 
object can be accurately estimated when the spectra of these 
illumination are known and the above conditions are satis- 
fied. 

In this embodiment, the color chips image is shot in 
advance under a known illumination source and stored. 
Therefore, it becomes possible to estimate the spectral 
reflectance of an object when shooting the object with use of 
the same color image input device for shooting the color 
chips, without shooting the color chips. 

Next, a color image processing apparatus according to the 
fourth embodiment of the present invention will now be 
described. 

As shown in FIG. 15, the color image processing appa- 
ratus includes a basis function storage unit 46 for storing 
basis functions of the spectral reflectance of human skin, an 
expansion coefficient setting unit 47 for setting an expansion 
coefficient corresponding to a basis function, a spectral 
reflectance calculating unit 48 for calculating a spectral 
reflectance, a spectral reflectance evaluating unit 49 for 
judging if a calculated spectral reflectance satisfies an evalu- 
ation condition, and a spectral reflectance storage unit 50 for 
storing a spectral reflectance which satisfies an evaluation 
condition. 

In the basis function storage unit 46, the basis functions 
of the spectral reflectance of the human skin, which has 
values at an interval of 1 nm within a wavelength range from 
380 nm to 780 nm. The number of basis functions stored in 
the basis function storage unit 46 is determined in accor- 
dance with the contribution rate of the basis function 
obtained by principal component analysis of a great number 
of the spectral reflectances of human skin measured in 
advance by the measurement instrument. The M -number of 
basis functions which are required to achieve a contribution 
rate of 99.9999% or more, are stored. 

The expansion coefficient setting unit 47 sets a expansion 
coefficient Cm (m-1 to M) which corresponds to a respec- 
tive one of the M- number of basis functions. 

The spectral reflectance calculating unit 48 calculates the 
spectral reflectance from the data input thereto, that is, the 
M-number of basis functions stored in the basis function 
storage unit 46 and the M-number of expansion coefficients 
set by the expansion coefficient setting unit 47. 

The spectral reflectance evaluation unit 49, to which the 
spectral reflectance calculated by the spectral reflectance 
calculating unit 48 is input, judges if the spectral reflectance 
satisfies the evaluation condition preset in the spectral 
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reflectance evaluation unit 49. If the condition is satisfied, coefficients for the bases of the color chips and the basis 

the spectral reflectance is output to the spectral reflectance functions of the spectral reflectances of human skin color are 

storage unit 50, whereas if the condition is not satisfied, the written as header data of shooting image data. The image 

expansion coefficient is reset in the expansion coefficient data or various data are read from the memory card 56 

setting unit 47. This operation is repeated until the 5 mounted in the personal computer 57 so as to perform the 

M- number of spectral reflectances are output. above-described process. The data are converted into RGB 

Next, the spectral reflectance evaluation unit 49 will now image data and displayed on the monitor 58. 

be described in detail with reference to FIG, 16. FIG. 18 is a diagram showing the concept of the format 

The spectral reflectance evaluation unit 49 includes a of the shooting image data stored in the memory card 56. 

non-negative condition evaluation unit 52 and a preparation 10 Color chart expansion coefficient data, chip spectral 

condition evaluation unit 53. With this structure, the spectral reflectance basis function data and object spectral reflec- 

reflectance calculated by the spectral reflectance calculating tance basis function data are recorded in the card as header 

unit 48 is input thereto, and whether or not the spectral data of one piece of image data. After the header data, the 

reflectance has a negative value is examined in the non- image data is recorded as RGB image data for each pixel, 

negative condition evaluation unit 52. 15 Each RGB channel is data of 1 byte. 

Here, in the case where there is no negative value within FIG. 19 is a diagram illustrating the concept of the color 

a wavelength range from 380 nm to 780 nm, the spectral conversion process. 

reflectance data is sent to the preparation condition evalu- The image data (RGB) signal read from the memory card 

ation unit 53, whereas in the case where there is a negative 56 to the personal computer 57 are converted into the 

value in the range, negative value wavelength data is output 20 spectral reflectance of the object from color chip expansion 

to the expansion coefficient setting unit 47. coefficient data recorded as image header data, chip spectral 

The preparation condition evaluation unit 53 evaluates reflectance basis function data, object spectral reflectance 

whether or not the spectral reflectance data sent from the basis function data, and chart shooting data, 

non-negative condition evaluation unit 52 can be designed The spectral reflectance is converted into XYZ values 

with available colorant which are stored as a data base in the 25 with use of observation-site illumination data and color 

colorant data storage unit 51 at an accuracy within an matching functions stored in the computer, and further 

allowable error range. converted into RGB image data to be input to a monitor, 

The colorant data storage unit 51 stores the diffusion using the monitor characteristics data, to be displayed as a 

coefficient and absorption coefficient of available colorant in color image on the CRT monitor. 

the form of data base. 30 As described above, the color image processing apparatus 

As the diffusion coefficient and absorption coefficient of of the present invention can estimate the spectral reflectance 

a colorant is input from the colorant data storage unit to the of an object from the object shooting signal Q and reference 

preparation condition evaluation unit, the spectral reflec- chart shooting signal G, shot by the color image input device 

tance closest to the one which can be designed from these under the same illumination condition, the color chips basis 

data is calculated. 35 functions vn(k) which can expand the spectral reflectances 

In the case where the spectral reflectance sent from the of every color chip of the reference chart pre -stored, the 

non-negative condition evaluation unit 52 satisfies an accu- expansion coefficient B of each color chip, and the basis 

racy within an allowable error range for the spectral reflec- function of the spectral reflectance of the object. In the 

tance data which can be designed by these colorant, the present invention, the conditions for being able to estimate 

spectral reflectance data is stored in the spectral reflectance 40 the spectral reflectance of an object without error are that the 

storage unit 50. In the case where the data cannot be basis functions of the spectral reflectance of the object can 

designed, it is output as design error data to the expansion be expanded by the color chip basis functions vn(X) of the 

coefficient setting unit 47. reference chart, and that the color image input device has a 

The expansion coefficient setting unit 47 resets the expan- certain number of bands, which is more than the number of 

sion coefficient Cm (m«l to M) on the basis of the data sent 45 basis functions of the spectral reflectances of the object, 

from the non-negative condition evaluation unit 52 and the With these conditions, even if the spectral sensitivities of 

preparation condition evaluation unit 53. the color image input device and the shooting-site illumi- 

When the color chip having a spectral reflectance pre- nation spectrum are not known, the spectral reflectance of 

pared in this embodiment is used as an object whose spectral the object can be estimated. 

reflectance is known in the first to third embodiments, it 50 Thus, according to the present invention, it is possible to 

becomes possible to estimate the spectral reflectance of an provide a color image processing apparatus for estimating 

object at higher accuracy. the spectral reflectance of the object from the color image 

Next, a color image processing apparatus according to the signal of the object, obtained by the input device even if the 

fifth embodiment of the present invention will now be spectral sensitivities of the input device and the shooting-site 

described. 55 illumination spectrum used when the color image signal is 

FIG. 17 schematically shows the structure of the embodi- input are not known, 

meat. Additional advantages and modifications will readily 

In this structure, an object (human image) shot by the occurs to those skilled in the art. Therefore, the invention in 

digital camera 54 and an image of the reference chart 55 its broader aspects is not limited to the specific details and 

prepared for human skin, which were taken under the 60 representative embodiments shown and described herein, 

shooting -site illumination are shot as one image. Accordingly, various modifications may be made without 

Then, the shooting image data is written in a memory card departing from the spirit or scope of the general inventive 

56 provided within the digital camera 54. In the memory concept as defined by the appended claims and their equiva- 

card 56, basis functions of spectral reflectances of the color lents. 

chips of the reference chart 55 which are read in advance in 65 What is claimed is: 

the digital camera 54 from the chart data record memory 1. A color image processing apparatus comprising an 

card attached to the reference chart 55, the expansion image processing device for processing an input image 
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signal of an object, in order to reproduce color data of the 
object, the image processing device further comprising: 
spectral reflectance calculating means including: 

a color chip shooting signal storage unit for calculating 
average values of shooting signal values correspond- 
ing to a plurality of color chips from a shooting 
image of a reference chart consisting of these color 
chips, for an appropriate region, and storing the 
average values as color shooting signals; 

a color chip expansion coefficient storage unit for 
storing expansion coefficients corresponding to basis 
functions of spectral reflectances of the color chips 
within the reference chart which are obtained in 
advance, as a first expansion coefficient; 

an object expansion coefficient calculating unit for 
calculating expansion coefficients of the object from 
an object shooting signal input from outside, a color 
chip shooting signal input from the color chip shoot- 
ing signal storage unit, and the first expansion coef- 
ficients input from the color chip expansion coeffi- 
cient storage unit; 

a color chip basis function storage unit for storing color 
basis functions vn(X) (n=l to 10); and 

a spectral reflectance calculating unit for calculating a 
spectral reflectance p(\) of the object from the 
expansion coefficients of the object calculated by the 
object expansion coefficient calculating unit and the 
color basis functions vn(X) (n=l to 10) of the object 
output from the color basis function storage unit, 
which outputs obtained spectral reflectance p(X); 
tristimulus values calculating means including: 

a color matching function data storage unit for storing 
color matching functions data in advance; and 

a tristimulus value operating unit for calculating tris- 
timulus values XYZ under observation-site illumi- 
nation for the object from the spectral reflectance 
p(X) of the object, and observation-site illumination 
spectrum data S(X) input from the observation-site 
illumination spectrum data storage unit provided 
outside; 

the tristimulus values calculating means outputting the 
tristimulus values XYZ; and 

RGB value calculating means including: 

a display device profile storage unit for storing char- 
acteristics data of the display device; and 
an output signal calculating unit for converting the 
tristimulus XYZ values into a display signals RGB, 
using the characteristics data of the display device; 

the RGB value calculating means outputting display sig- 
nals RGB to the display device. 

2. A color image processing apparatus according to claim 
1, wherein the object expansion coefficient calculating unit 
further comprises: 

an inverse matrix operating unit for calculating an inverse 
matrix from the first expansion coefficients input from 
the color chip expansion coefficient storage unit; 

a first matrix multiplication unit for calculating a first 
matrix by multiplying the inverse matrix with the color 
chip shooting signal input from the color chip shooting 
signal storage unit; 

a second inverse matrix operating unit for calculating a 
second inverse matrix which is an inverse matrix to the 
first matrix calculated; and 

a matrix multiplication unit for calculating a second 
expansion coefficient by multiplying the second inverse 



10 



15 



20 



25 



40 



45 



60 



65 



matrix output from the inverse matrix operating unit as 
the reference object shooting signal is input thereto. 

3. A color image processing apparatus according to claim 
1, wherein the spectral reflectance calculating unit further 
comprises: 

a basis coefficient multiplication unit for multiplying a 
second expansion coefficient C input from the reference 
object expansion coefficient calculating unit with a 
color chip basis functions vn(X) corresponding to the 
second expansion coefficients, input from the color 
chip basis function storage unit; and 

a basis function integrating unit for calculating the spec- 
tral reflectance by adding up the data input from the 
basis coefficient multiplying unit with regard to all the 
basis functions cnvn(X) (n-1 to 10) for each wave- 
length. 

4. A color image processing apparatus according to claim 
1, wherein the spectral reflectance calculating unit further 
comprises: 

a reference object basis function storage unit for storing 
object basis functions el(X); and 

an object basis expansion coefficient calculating unit for 
calculating third expansion coefficients from the color 
chip basis functions input from the color chip basis 
function storage unit and the object basis functions 
input from the object basis function storage unit, and 
outputting the third expansion coefficients to the object 
expansion coefficient calculating unit. 

5. A color image processing apparatus for processing an 
input image signal of an object, in order to reproduce color 
data of the object, the apparatus comprising: 

color image input means for obtaining a color image 
signal of the object and a color image signal of a 
reference matter having a plurality of colors, in lighting 
for photographing; 

color estimating means for estimating the color data of the 
object from the color image signal of the object; and 

color image output means for reproducing the color image 
of the object based on the estimated color data of the 
object; 

wherein the color estimating means further comprises: 
matrix calculation means for calculating a matrix which 
provides a color image signal of the lighting for 
photographing, to a coefficient corresponding to a 
basis function of a spectral reflectance of the refer- 
ence matter, by using the color image signal of the 
reference matter and the spectral reflectance of the 
reference matter; 
spectral reflectance calculation means for estimating a 
spectral reflectance of the object from the calculated 
matrix, the basis function of the spectral reflectance 
of the reference matter and the color image signal of 
the object; and 
color data calculating means for calculating color data 
of the object from the estimated spectral reflectance. 

6. The color image processing apparatus according to 
claim 5, wherein the spectral reflectance of the object whose 
spectral reflectance is known, is calculated by spectral 
reflectance designing means which comprise: 

basis function storage means for storing a plurality of 

basis functions of spectral reflectances; 
expansion coefficient adjusting means for setting values 

of expansion coefficients; 
spectral reflectance calculating means for calculating a 

spectral reflectance from the basis functions and the 

expansion coefficients; and 
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spectral reflectance evaluation means for evaluating the 
spectral reflectance. 

7. The color image processing apparatus according to 
claim 5, wherein the spectral reflectance of the object 
includes a basis function of a spectral reflectance and an $ 
expansion coefficient for each basis function of a spectral 
reflectance of the object in advance. 

8. The color image processing apparatus according to 
claim 5, wherein the basis function of the spectral reflec- 
tance of the object is expressed by a linear combination of 
the basis function of the spectral reflectance of the object. 

9. The color image processing apparatus according to 
claim 5, wherein a product of the basis function of the 
spectral reflectance of the object and the illumination spec- 
trum used when the color image signal of the object is 
obtained, is expressed by a linear combination of a product 15 
of the basis function of the spectral reflectance of the object 
and the illumination spectrum used when the color image 
signal of the reference object is obtained. 

10. The color image processing apparatus according to 
claim 5, wherein the color estimating means reproduces the 20 
color data of the object by calculating a calorimetric value 

of the object using data on a spectrum of the lighting for 
photographing in an observation circumstance for reproduc- 
ing the color data of the object. 

U. The color image processing apparatus according to 25 
claim 5, wherein the color image signal input means 
includes at least four independent spectral sensitivity char- 
acteristics. 

12. A color image processing apparatus for processing an 
input image signal of an object, in order to reproduce color 
data of the object, the apparatus comprising: 

color image input means for obtaining a color image 
signal of the object and a color image signal of a 
reference matter having a plurality of colors, in lighting 
for photographing; 
color estimating means for estimating the color data of the 

object from the color signal of the object; and 
color image output means for reproducing the color image 
of the object based on the estimated color data of the 
object; 40 
wherein the color estimating means further comprises: 
matrix calculation means for calculating a matrix which 
provides a color image signal in the lighting for 
photographing, to a coefficient corresponding to a 
basis function of a spectral reflectance of the refer- 45 
ence matter, by using the color image signal of the 
reference matter, the spectral reflectance of the ref- 
erence matter, and a basis function of a spectral 
reflectance of the object; 
spectral reflectance calculation means for estimating a 50 
spectral reflectance of the object from the calculated 
matrix, the basis function of the spectral reflectance 
of the reference matter and the color image signal of 
the object; and 
color data calculating means for calculating color data 55 
of the object from the estimated spectral reflectance. 

13. The color image processing apparatus according to 
claim 12, wherein a spectral reflectance of the object whose 
spectral reflectance is known, is calculated by spectral 
reflectance designing means which comprise: 60 

basis function storage means for storing a plurality of 

basis functions of spectral reflectances; 
expansion coefficient adjusting means for setting values 

of expansion coefficients; 
spectral reflectance calculating means for calculating a 65 

spectral reflectance from the basis functions and the 

expansion coefficients; and 
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spectral reflectance evaluation means for evaluating the 
spectral reflectance. 

14. The color image processing apparatus according to 
claim 12, wherein the spectral reflectance of the object 
includes a basis function of a spectral reflectance and an 
expansion coefficient for each basis function of a spectral 
reflectance of the object in advance. 

15. The color image processing apparatus according to 
claim 12, wherein the basis function of the spectral reflec- 
tance of the object is expressed by a linear combination of 
the basis function of the spectral reflectance of the object. 

16. The color image processing apparatus according to 
claim 12, wherein a product of the basis function of the 
spectral reflectance of the object and the illumination spec- 
trum used when the color image signal of the object is 
obtained, is expressed by a linear combination of a product 
of the basis function of the spectral reflectance of the object 
and the illumination spectrum used when the color image 
signal of the reference object is obtained. 

17. The color image processing apparatus according to 
claim 12, wherein the color estimating means reproduces the 
color data of the object by calculating a colorimetric value 
of the object using data on a spectrum of the lighting for 
shooting in an observation circumstance for reproducing the 
color data of the object. 

18. The color image processing apparatus according to 
claim 12, wherein the color image signal input means has at 
least four independent spectral sensitivity characteristics. 

19. A color image processing apparatus for processing an 
input image signal of an object, in order to reproduce color 
data of the object, the apparatus comprising: 

color image input means for obtaining a color image 
signal of the object in lighting for photographing and a 
color image signal of a reference matter having a 
plurality of colors in reference lighting for photograph- 
ing; 

color estimating means for estimating the color data of the 
object from the color signal of the object; and 

color image output means for reproducing the color image 
of the object based on the estimated color data of the 
object, 

wherein the color estimating means further comprises: 
matrix calculation means for calculating a matrix which 
provides a color image signal in the lighting for 
photographing, to a coefficient corresponding to a 
basis function of a spectral reflectance of the refer- 
ence matter, by using the color image signal of the 
reference matter, the spectral reflectance of the ref- 
erence matter, a spectrum of the reference lighting 
for shooting, a basis function of a spectral reflectance 
of the object, and a spectrum of the lighting for 
photographing; 
spectral reflectance calculation means for estimating a 
spectral reflectance of the object from the calculated 
matrix, the basis function of the spectral reflectance 
of the reference matter and the color image signal of 
the object; and 
color data calculating means for calculating color data 
of the object from the estimated spectral reflectance. 

20. The color image processing apparatus according to 
claim 19, wherein a spectral reflectance of the object whose 
spectral reflectance is known, is calculated by spectral 
reflectance designing means which comprise: 

basis function storage means for storing a plurality of 

basis functions of spectral reflectances; 
expansion coefficient adjusting means for setting values 

of expansion coefficients; 
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spectral reflectance calculating means for calculating a 
spectral reflectance from the basis functions and the 
expansion coefficients; and 

spectral reflectance evaluation means for evaluating the 
spectral reflectance. 5 

21. The color image processing apparatus according to 
claim 19, wherein the spectral reflectance data of the object 
includes a basis function of a spectral reflectance and an 
expansion coefficient for each basis function of a spectral 
reflectance of the object in advance. 10 

22. The color image processing apparatus according to 
claim 19, wherein the basis function of the spectral reflec- 
tance of the object is expressed by a linear combination of 
the basis function of spectral reflectance of the object. 

23. The color image processing apparatus according to 15 
claim 19, wherein a product of the basis function of the 
spectral reflectance of the object and the illumination spec- 
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trum used when the color image signal of the object is 
obtained, is expressed by a linear combination of a product 
of the basis function of the spectral reflectance of the object 
and the illumination spectrum used when the color image 
signal of the reference object is obtained. 

24. The color image processing apparatus according to 
claim 19, wherein the color estimating means reproduces the 
color data of the object by calculating a calorimetric value 
of the object using data on a spectrum of the lighting for 
shooting in an observation circumstance for reproducing the 
color data of the object. 

25. The color image processing apparatus according to 
claim 5, wherein the color image signal input means has at 
least four independent spectral sensitivity characteristics. 

* * * * * 
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